Condensing Graphical Representations of Media Clips in a Composite Display Area of a Media-Editing Application

ABSTRACT

Some embodiments provide a computer readable medium storing a media editing application for creating multimedia presentations. The application includes a graphical user interface (GUI). The GUI includes a composite display area for displaying graphical representations of a set of media clips that are part of a composite presentation. Each graphical representation of a particular media clip is assigned to a particular row in the composite display area, where each row corresponds to a particular track in the composite presentation. Some embodiments of the GUI include a compression tool for assigning the graphical representations to new rows so as to reduce blank space in the composite display area, where the assignment of the graphical representations to new rows eliminates the correspondence between the rows and the tracks. Some embodiments include a collapsing tool for reducing a size of graphical representations in the composite display area.

FIELD OF THE INVENTION

The invention is directed towards the presentation of a compositedisplay area in a media-editing application. Specifically, the inventionis directed towards methods for condensing the display of such acomposite display area.

BACKGROUND OF THE INVENTION

Media editing applications allow users to create composite multimediapresentations (e.g., movies) based on several multimedia clips, such asaudio and video clips. The graphical user interface (GUI) of such amedia editing application will often include a composite display areathat includes several tracks that span a timeline. On the tracks, thecomposite display area displays rectangles or other shapes thatrepresent the clips used to create the multimedia presentation.

Often, the composite display area cannot display all the tracks. Only aparticular portion of the GUI of the application is reserved for thecomposite display area, and this may not be enough room to display allof the tracks. Thus, a user must scroll to view some of the tracks, butthis will move other tracks out of the display. Once a particular numberof tracks is exceeded, the application can no longer concurrentlydisplay all of the tracks in the display area. This can make itdifficult for a user to accurately line up clips that are in tracks farapart. Thus, there is a need for a media editing application that allowsa user to condense the media clips in a composite display area in oneway or another.

SUMMARY OF THE INVENTION

Some embodiments of the invention provide a media editing applicationthat allows a user to visually condense graphical representations ofmedia clips that the application combines to create a composite mediapresentation. The media editing application of some embodiments includesa display area for displaying the composite presentation that theapplication creates by compositing several media clips (e.g., audioclips, video clips). The media editing application of some embodimentsalso includes a composite display area (e.g., an area with multipletracks that span a timeline) for displaying the graphicalrepresentations of the media clips that are part of the compositepresentation.

Each graphical representation is displayed in a particular row and spansa particular portion of the timeline. In some embodiments, the displayof a graphical clip representation in a particular row indicates theassignment of the corresponding media clip to a track used when theapplication creates the composite presentation. To condense thegraphical representations in the composite display area, the applicationof some embodiments includes a compression tool for causing thegraphical representations to be moved together across the rows in orderto reduce blank space in the composite display area. Alternatively orconjunctively, the application of some embodiments includes a collapsingtool for reducing the size of the graphical representations in thecomposite display area without moving the graphical representations fromtheir assigned tracks.

To compress the clip representations, some embodiments identify new rowassignments for the clip representations in the composite display areathat do not necessarily indicate the assignment of the correspondingmedia clip to a particular data track. The compression tool thencalculates speeds at which each clip representation should be moved fromits initial row to its newly assigned row, and moves the cliprepresentations to the newly assigned rows in unison.

In identifying the new row assignments, some embodiments attempt to moveall of the clip representations towards a particular destination row(e.g., the top row or bottom row). For each particular cliprepresentation, the compression tool identifies the row closest to thedestination row to which the clip representation can be moved withoutmoving past any other clip representations that span any portion of thetimeline overlapping with the portion spanned by the particular cliprepresentation. Thus, some clip representations that are initiallyassigned to the same row (and thus, whose corresponding media clips areassigned to the same track) may be moved to different rows depending onthe presence of clip representations that overlap them in the timeline.

To calculate the speeds at which the clip representations should bemoved, the compression tool of some embodiments identifies the relativedistances that the different clip representations are to be moved andthe amount of time over which the movement is to be displayed. Thespeeds are then calculated assuming that all of the clip representationsshould begin and end their movement at the same time. Thus, a cliprepresentation that is moving three rows will move three times as fastas a clip representation that is moving one row in some embodiments. Themedia editing application then displays the movement of the cliprepresentations to their newly assigned rows in the composite displayarea.

To collapse the clip representations in the composite display area, someembodiments shrink the size of the selected clip representations and therows in which the selected clip representations are displayed in adirection orthogonal to the timeline. That is, if the timeline isdisplayed horizontally, the selected clip representations and rows arecollapsed so as to take up less vertical space in the composite displayarea. This enables more rows to be displayed in the composite displayarea at one time. Unlike the compression tool, the collapsing toolmaintains the association of a clip representation with the track towhich the corresponding media clip is assigned.

Some embodiments allow a user to apply the compression tool orcollapsing tool (collectively, the condensing tools) to only a subset ofthe clip representations in the composite display area. Some embodimentsarrange the clip representations into groups and sub-groups (andsub-groups of sub-groups, etc.) and enable a user to select one or moregroups before applying one of the condensing tools. In some embodiments,each row in the composite display area is assigned to a particular groupand/or sub-group, and all of the clips in a particular row are in theparticular row's group and/or sub-group. When one of the condensingtools is applied with a particular group or sub-group selected, someembodiments only apply the features of that condensing tool to the cliprepresentations in the particular group or sub-group.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth in the appendedclaims. However, for purpose of explanation, several embodiments of theinvention are set forth in the following figures.

FIG. 1 illustrates the application of a compression tool of someembodiments to a graphical user interface of a media editingapplication.

FIG. 2 illustrates the application of a collapsing tool of someembodiments to a graphical user interface of a media editingapplication.

FIG. 3 illustrates a graphical user interface of a media-editingapplication of some embodiments.

FIG. 4 illustrates a composite presentation window that shows theentirety of a media presentation.

FIG. 5 conceptually illustrates a process of some embodiments formodifying the vertical size of clip shapes in one or more groups.

FIGS. 6-9 illustrate the selection of two sub-groups within a compositedisplay area of a media-editing application and the application of acollapsing tool to the selected sub-groups according to someembodiments.

FIGS. 10-12 illustrate the selection of a sub-group and the applicationof an expansion tool to the selected sub-group.

FIG. 13 conceptually illustrates a process of some embodiments forapplying a compression feature

FIGS. 14 and 15 illustrate the application of a compression feature ofsome embodiments to clip shapes in sub-groups within a composite displayarea of a media-editing application.

FIGS. 16 and 17 illustrate the application of a compression feature ofsome embodiments to clip shapes across sub-groups within a compositedisplay area of a media-editing application.

FIGS. 18 and 19 illustrate the application of a compression feature ofsome embodiments to a composite display area in which track lines aremaintained.

FIG. 20 conceptually illustrates a process 2000 of some embodiments forassigning clip shapes to new rows when applying the compression featureto clip shapes in a composite display area.

FIG. 21 illustrates a GUI for a media-editing application that includesa composite display area.

FIGS. 22A-22D illustrate a row assignment process of some embodiments asapplied to the composite display area of the GUI of FIG. 21.

FIG. 23 illustrates the animation of clip shapes in the compositedisplay area of the GUI of FIG. 21.

FIG. 24 conceptually illustrates the software architecture of amedia-editing application of some embodiments.

FIG. 25 conceptually illustrates a process of some embodiments formanufacturing a computer readable medium that stores a computer program.

FIG. 26 illustrates a detailed view of a media-editing application ofsome embodiments.

FIG. 27 conceptually illustrates a computer system with which someembodiments of the invention are implemented.

DETAILED DESCRIPTION OF THE INVENTION

In the following description, numerous details are set forth forpurposes of explanation. However, one of ordinary skill in the art willrealize that the invention may be practiced without the use of thesespecific details. For instance, many of the examples below display atimeline running horizontally with tracks or rows stacked on top of eachother vertically. One of ordinary skill will recognize that a timelinecould be displayed vertically and tracks or columns then lined up nextto each other horizontally.

Some embodiments provide a media-editing application for creating amultimedia presentation (e.g., a movie) by compositing severalmultimedia clips (e.g., audio clips, video clips, etc.). Themedia-editing application of some embodiments provides (1) a compositedisplay area for displaying a set of clip shapes representing a set ofmultimedia clips that are part of the composite presentation and (2) oneor more condensing tools (i.e., a compression tool and/or a collapsingtool) for visually condensing the clip shapes in the composite displayarea. In some embodiments, the set of clip shapes are arranged intodifferent groups and sub-groups in the composite display area and thecondensing tools can be applied to one or more groups of clip shapesrather than the entire set of clip shapes.

For some embodiments of the invention, FIG. 1 illustrates a graphicaluser interface (“GUI”) 100 of a media editing application with such acompression tool. Specifically, FIG. 1 illustrates the GUI at twodifferent stages, a first stage 105 before the application of thecompression tool and a second stage 110 after the application of thecompression tool.

As shown in FIG. 1, the GUI 100 includes a preview display area 115, acomposite display area 120, and a compression UI item 125. The previewdisplay area 115 displays a preview of a composite presentation that theapplication creates by compositing several media clips. The compositedisplay area 120 provides a visual representation of the compositepresentation being created by the user. Specifically, it displays one ormore clip shapes representing one or more media clips that are part ofthe composite presentation.

Composite display area 120 is an area that includes seven rows that spana timeline 135. In some embodiments, the timeline 135 indicates the timeduring the composite presentation that particular media clips will bepresented within the presentation. Each of the seven rows includes alabel that indicates a track with which the media clips represented in aparticular row are associated. Each row is associated with a particularset of vertical coordinates. The seven rows are assigned to two groups,a video group 140 and an audio group 150. Each of the groups includestwo sub-groups. Video sub-group A includes clip shapes 141 and 142 whilevideo sub-group B includes clip shapes 143-146. Audio sub-group Aincludes clip shapes 151 and 152 while audio sub-group B includes clipshapes 153-157. In this example, the video and audio groups areindicated by group headers while the sub-groups are indicated in thetrack labels (e.g., “V-A1” indicates video sub-group A).

The compression UI item 125 is a conceptual illustration of one or moreUI items that causes the media editing application to implement itscompression feature for moving the clip shapes together across the rowsin order to reduce blank space in the composite display area. Differentembodiments of the invention implement the compression item differently.Some embodiments implement it as a UI button, others as a command thatcan be selected in a pull-down, drop-down, or other type of menu, andstill others as a command invoked through one or more keystrokeoperations. Yet other embodiments allow the user to access thecompression feature through multiple different UI items.

The operation of the compression feature will now be described byreference to the GUI during the two stages 105 and 110 illustrated inFIG. 1. In the first stage 105, the composite display area 120 displaysa graphical representation of a portion of a composite presentation thatincludes the several clip shapes described above along the timeline 135.A user could have added these clip shapes to the composite display areain order to define the composite presentation in a current editingsession or the user could have opened a composite project defined in aprevious editing session. In the first stage, each clip shape isdisplayed in a row that indicates the assignment of the correspondingmedia clip to a track used when the application creates the compositepresentation.

The second stage 110 illustrates the result of a user selecting thecompression UI item 125 to cause the application to compress the clipshapes in the composite display area.

When the compression feature is applied, the application moves the clipstogether in order to reduce blank space in the composite display area.In some embodiments, when there are two or more groups of media clips,compression is applied to each of the groups separately, as shown. Inthe case illustrated in FIG. 1, the upper group (video) is compresseddownwards while the lower group (audio) is compressed upwards. Otherembodiments implement the compression differently. For instance, someembodiments compress all of the clip shapes in one direction (e.g.,towards the top of the composite display area). Some such embodimentscompress each group of clip shapes separately in the same directionwhile other embodiments compress all of the clip shapes together.

The particular application of the compression feature illustrated inFIG. 1 causes video clip shapes 141 and 142 to move downwards such thatthey are on the same row as clip shape 143. Similarly, audio clip shapes153, 154, and 157 are moved upwards onto the same row as clip shapes 151and 152, while clip shape 156 is moved upwards onto the same row as clipshape 155. While the compression process does not modify the location ofa clip shape with reference to the timeline (i.e., the horizontaldirection in GUI 100), it does disassociate the rows from the trackassignments, in that two clip shapes whose corresponding media clips areassigned to the same track may be on the same row. Doing so does notmean that the corresponding media clip is actually assigned to adifferent track (e.g., the media clip corresponding to clip shape 141 isstill assigned to video track A1, not to track B1). Accordingly, someembodiments no longer display the track names as labels, as shown. Somesuch embodiments also remove the track lines from the composite displayarea as well. Other embodiments, however, keep the track names anddisplay the track lines as moving up and down in the composite displayarea with the clip shapes. For example, in such embodiments, the trackline between V-A1 and V-A2 would diagonal downwards sharply, run belowclip 141, then diagonal upwards to run above clips 143 and 142.Different applications of the compression tools of some embodiments willbe described in greater detail below in Section III and IV.

As mentioned above, in addition to or in place of a compression tool,some embodiments include a collapsing tool. FIG. 2 illustrates a GUI 200of a media editing application of some embodiments with such acollapsing tool. Specifically, FIG. 2 illustrates the GUI at twodifferent stages, a first stage 205 before the application of thecollapsing tool and a second stage 210 after the application of thecollapsing tool.

Like the GUI 100 of FIG. 1, the GUI 200 of FIG. 2 includes a previewdisplay area 215 and a composite display area 220. GUI 200 also includesa collapsing UI item 225. The composite display area 220 is arrangedsimilarly, with video clips 241-246 split into two subgroups and audioclips 251-257 split into two subgroups. The clip shapes are displayedalong rows 230 that span a timeline 235.

The collapsing UI item 225 is a conceptual illustration of one or moreUI items that causes the media editing application to implement itscollapsing feature for reducing the size of the clip shapes. Differentembodiments of the invention implement the collapsing UI itemdifferently. Some embodiments implement it as a UI button, others as acommand that can be selected in a pull-down, drop-down, or other type ofmenu, and still others as a command invoked through one or morekeystroke operations. Yet other embodiments allow the user to access thecollapsing feature through multiple different UI items.

In the first stage 205 of FIG. 2, the composite display area displays agraphical representation of a portion of a composite presentation thatincludes the several clip shapes described above along the timeline 235.Each clip shape is displayed in a row that indicates the assignment ofthe corresponding media clip to a track used when the applicationcreates the composite presentation.

The second stage illustrates the result of a user selecting thecollapsing UI item 225 to cause the application to collapse the videoclip shapes. The collapsing feature of some embodiments may be appliedto either groups or sub-groups in order to de-emphasize those groups orsub-groups. In the particular case shown in FIG. 2, the entire videogroup is collapsed.

In some embodiments, collapsing a group of clip shapes entails reducingthe size of the clip shapes in the direction orthogonal to the timeline.In this case, the clip shapes are reduced in size vertically because thetimeline is in the horizontal direction. This way, a user can stilleasily recognize the portion of the timeline spanned by each clip shape.The collapsing feature of some embodiments also leaves each clip shapein the same row that it was in prior to the application of the feature.As such, the rows maintain their association with particular tracks uponthe application of the collapsing feature. However, some embodiments nolonger display the track names, as shown in stage 210. Reducing the sizeof some of the clip shapes provides more space in the composite displayarea for other clip shapes. As shown, collapsing the video group revealsthat there are more audio clip shapes 261-265 in a third audiosub-group.

In the examples described above, both the compression and collapsingfeatures are shown as applied to groups of clips in a composite displayarea. One of ordinary skill in the art will recognize that someembodiments include these condensing tools without grouping the clipshapes in the composite display area. Thus, some embodiments apply thecompression feature to move together all of the clip shapes in thecomposite display area, or apply the collapsing feature to reduce thesize of all of the clip shapes in the composite display area.

Further details regarding the media-editing application of someembodiments are described in the following Section I. This section alsoincludes details on the grouping feature of some such applications.Section II then describes the collapsing feature of some embodiments, aswell as an expansion feature. Section III follows this with furtherdetails about the compression feature of some embodiments. Section IVelaborates on the row assignment aspect of the compression process.Section V then describes the software architecture of some embodimentsof the media-editing application and Section VI provides a descriptionof one computer system with which some embodiments of the invention areimplemented.

I. Media-Editing Application

As mentioned above, some embodiments of the invention provide amedia-editing application that includes various tools for visuallycondensing graphical representations of media clips used to create acomposite media presentation. FIG. 3 illustrates a graphical userinterface (GUI) 300 of such a media-editing application. The GUI 300includes a media library 305, a preview display area 310, a set of tools315, and a composite display area 320.

The media library 305 is a region in the GUI that includes a group ofselectable media clips. A user can select a media clip in order to addthe media clip or a portion of the media clip to the media presentationthat the user is compositing with the application. In some embodiments,as shown, the media clips are represented as thumbnails that can beselected and added to the composite display area 320 (e.g., through adrag-and-drop operation or a menu selection operation). In someembodiments, the media library may include video clips, audio clips,text overlays, pictures, or other media.

The preview display area 310, as mentioned above, displays a preview ofthe composite presentation that the application creates by compositingthe media clips added by the user. As shown, some embodiments includevarious video player tools, such as a play button, pause button, fastforward, rewind, etc. in order to further aid the user in viewing thepreview of the media presentation.

The set of tools 315 is representative of various selectable toolspresent in the media editing application of some embodiments. In someembodiments, the set of tools includes various media editing tools. Forinstance, set of tools 315 includes a roll tool 325 and a slip tool 330for performing various types of trim edits on the clips in the compositedisplay area. Some embodiments include other editing tools as well, suchas a slide tool, ripple tool, blade tool, etc. The set of tools 315 alsoincludes various tools for affecting the display of the compositedisplay area, such as collapsing tool 335, compression tool 340, andexpansion tool 345. The operation of these tools will be described indetail in the sections that follow.

The composite display area 320 provides a visual representation of thecomposite presentation being created by the user. Specifically, itdisplays one or more geometric shapes that represent media clips (i.e.,clip shapes) that are part of the composite presentation. Compositedisplay area 320 includes timeline 350 for indicating the time withinthe media presentation to which horizontal positions in the compositedisplay area correspond. Each particular media clip is placed on aparticular row in the composite display area and spans a particularportion of the timeline in some embodiments. Each row spans a particularset of y-coordinates.

In some embodiments, each row is associated with a track that is used bythe editing application to create the composite presentation. Differenttracks are used differently in some embodiments. For example, someembodiments will use all audio tracks at once. On the other hand, ifthere is more than one primary video track, some embodiments have ahierarchy of which video track supersedes the other(s). That is, ifthere is a first clip assigned to a first video track at a particulartime and a second clip assigned to a second video track at theparticular time, some embodiments will only incorporate one of the clipsinto the media presentation for that particular time.

In some embodiments, as shown, the clip shapes in the composite displayarea are arranged down into groups. For instance, some embodiments groupthe clip shapes based on the type of media. The clip shapes in compositedisplay area 320 are arranged into two primary groups, each of which hasmultiple subgroups. The video group 355 includes sub-groups main edit356, supers 357, and video effects 358. The audio group 360 includessub-groups main audio 361, dialog 362, audio effects 363, and music 364.In the example shown, there are no labels for the two groups 355 and360, but they are separated by the thick line 365. The track labels ineach group also begin with a “V” for the video tracks and an “A” for theaudio tracks. The sub-groups, on the other hand, each have a label offto the left side to indicate which tracks belong to which sub-group.

The groups and sub-groups shown in composite display area 320 are merelyone example of the groups and sub-groups that could be provided in amedia-editing application. Some embodiments of the media-editingapplication have default group and sub-group settings that areautomatically provided to a user In some embodiments, a user can alsodefine new groups and sub-groups or delete groups and sub-groups byusing various user interface tools.

In some embodiments, some of the tools 315 can be applied to individualgroups and/or sub-groups as well as to the entire composite displayarea. For instance, some embodiments allow a user to select one or moresub-groups and then apply one of the UI tools 335, 340, or 345 to affectonly the tracks and clip shapes in the selected group.

Different groups can have different numbers of tracks, and thus havedifferent numbers of rows in the composite display area. In the exampleshown in FIG. 3, the main edit sub-group 356 only has two tracks,whereas the main audio sub-group 361 has four tracks. In someembodiments, the number of tracks in any particular sub-group is notlimited. As more tracks are added to the media presentation, a smallerpercentage of the total rows will be visible at any one time in thecomposite display area. In GUI 300, not all of the rows can be displayedat one time in composite display area 320, as at least one row in themusic sub-group 364 is cut off. A user can use scrollbar 370, or asimilar UI item, to scroll down to see the bottom rows in the compositedisplay area.

In some embodiments, the media-editing application keeps track of theentire composite media presentation, including that which is notpresently displayed in the composite display area, via a compositepresentation window. Each clip shape is assigned a location in thecomposite presentation window that spans x-coordinates (on the timeline)and y-coordinates (row assignment and display size for the row). Thecomposite display area displayed in the GUI is then a portion of thisoverall composite presentation window that is defined by a span ofx-coordinates and y-coordinates.

FIG. 4 illustrates a complete composite presentation window 400 thatshows the entirety of the media presentation from FIG. 3. FIG. 4 alsoillustrates window 410 as a dashed line that indicates the portion ofthe presentation window 400 that is presently displayed in compositedisplay area 320. Moving the scrollbars in the GUI 300 willcorrespondingly move window 410, thereby changing what is displayed incomposite display area 320. The following sections also present variousmethods to modify how much of the overall presentation can be viewed atone time in the composite display area.

II. Collapsing and Expanding Clip Shapes

As described above, some embodiments provide UI tools for collapsingand/or expanding clip shapes. In some embodiments, these collapsing andexpanding features can be applied to different groups and/or sub-groupsin the composite display area. FIG. 5 conceptually illustrates a process500 of some embodiments for modifying the vertical size of clip shapesin one or more groups. Process 500 will be described by reference toFIG. 6-12.

FIGS. 6-12 illustrate a GUI 600 for a media-editing application. Similarto the GUI 300 of FIG. 3, GUI 600 includes a media library 605, apreview display area 610, and a composite display area 615. GUI 600 alsoincludes a collapsing tool 620 and an expansion tool 625.

As shown in FIG. 5, process 500 begins by displaying (at 505) acomposite display area that includes several clip shapes arranged ingroups. Composite display area 615 of GUI 600 is an example of such acomposite display area. Like GUI 300, GUI 600 includes two primarygroups of tracks (video and audio) that are arranged into sevensub-groups (three video sub-groups and four audio sub-groups).

The process next receives (at 505) a selection of one or more groups ofclip shapes. In some embodiments, a selection of a group selects therows of the group, which in turn each include a set of clip shapes. Insome embodiments, a user moves a cursor over the group label, thenpresses and releases a selection button on a cursor controller (e.g., aleft mouse button) in order to select a group. In some embodiments, auser touches a touchscreen where the group label is displayed in orderto select a group. Other selection mechanisms, such as using keyboardinput, are possible as well. Some embodiments include a checkbox orsimilar UI feature for each group that a user can check or uncheck toselect or deselect a group.

The process then highlights (at 515) the selected groups in order toindicate this selection. Some embodiments highlight the group label andsome embodiments highlight the labels of each of the rows in theselected group (i.e., the track names). Other areas may be highlightedas well in order to convey to the user that a particular group isselected. The highlighting can be a change in color, a pattern displayedover the highlighted portion, or some other visual mechanism.

FIGS. 6-8 illustrate the selection of two sub-groups within thecomposite display area 615. In addition to the features of the GUI 600that are mentioned above, these figures illustrate a cursor 630. In FIG.6, cursor 630 is displayed over the label for the supers video sub-group635.

FIG. 7 illustrates the result of user interaction to select the superssub-group 635 (e.g., the user pressing and releasing a mouse button withthe cursor in the location shown in FIG. 6). The labels for the threerows in the supers sub-group, rows 705, 710, and 715 (V3, V4, and V5),are now highlighted. In the example shown, only the row labels arehighlighted and not the group label itself. In some embodiments, a usercan also select individual rows, or select a group and then deselect oneof the individual rows by pressing and releasing a mouse button with thecursor over the row label (or similar input).

FIG. 7 also illustrates the cursor 630 over the label for the dialogaudio sub-group 720. FIG. 8 illustrates the GUI 600 after userinteraction to select the dialog sub-group 720. The labels for the threerows in the dialog audio sub-group, rows 805, 810, and 815 (A5, A6, andA7) are now highlighted.

Returning to FIG. 5, process 500 next receives (at 520) input tocollapse or expand the selected clip shapes. In some embodiments, theseoptions are provided as user interface items such as items 620 and 625.The items may be selectable UI buttons, commands in a pull-down,drop-down, or other type of menu, or other types of UI items. In someembodiments, the options may also be invoked by keyboard input.

After receiving input to collapse or expand the selected clip shapes,the process calculates (at 525) new locations for the clip shapes in thecomposite display area. In some embodiments, this entails calculatingnew locations in the composite presentation window (such as compositepresentation window 400 of FIG. 4). Once the new locations in thecomposite presentation window are calculated for the clip shapes, thenthe new locations in the displayed composite display area are only amatter of determining what is within the display window (e.g., window410).

The new locations for the clip shapes are due to the changing in size ofthe clip shapes. In some embodiments, the collapsing tool causes theselected clip shapes to be reduced in size vertically and the expansiontool causes the selected clip shapes to be enlarged in size vertically.This means that the rows to which the selected clip shapes are assignedmust be reduced or enlarged in size vertically, and the rows aboveand/or below must move accordingly.

After calculating the new positions, the process displays (at 530) theclip shapes in their new positions, and then ends. FIGS. 8 and 9illustrate the collapsing feature as applied to the two selected groups,supers video sub-group 635 and dialog audio sub-group 720. In FIG. 8,with the two groups selected, the user has moved the cursor 630 overcollapsing tool 620.

FIG. 9 illustrates the result of the user selecting the application ofcollapsing tool 620. The clip shapes in the supers sub-group 635 anddialog sub-group 720 have been reduced in size vertically by a factor ofapproximately one-fourth along with the row height for the rows in thesesub-groups. Although the clip shapes reduce in size vertically, the rowassignments of the clip shapes are not lost because the rows are reducedin size correspondingly. Thus, the clip shapes are still displayed in arow that corresponds to the track to which the clip shapes'corresponding media clips are assigned. Furthermore, because the size ofthe clip shapes horizontally is not affected, the association of eachclip shape with a particular portion of the timeline is not lost. Infact, in some embodiments, the user may still perform editing operationson the collapsed clip shapes, although doing so with precision may bemore difficult due to the reduced size. As shown, however, someembodiments remove the row labels upon collapsing the clip shapes due tothe reduced size. In fact, some embodiments change the orientation ofthe group label from vertical (as in FIG. 8) to horizontal (as in FIG.9), and remove the boundary between the group label and the row labels.

In order to change from the display in FIG. 8 to the display in FIG. 9,the editing application must calculate the new locations of all of theclip shapes. In embodiments that begin at the top of the compositedisplay area and work downwards, the clip shapes in the main editsub-group 905 are not affected at all, because they are above the firstcollapsed sub-group. However, new vertical spans for each of the clipshapes in the supers sub-group 635 must be calculated based on thereduction factor. In turn, clip shapes in the video effects sub-group910 and main audio sub-group 920 are moved upwards now that the size ofthe clip shapes above is reduced. Calculations for the dialog sub-group720 must factor in moving the group upwards due to the reduction in sizeof the supers sub-group as well as the reduction in size of the dialogsub-group itself. The positions of the clip shapes in audio effectssub-group 920 and music sub-group 925 can then be calculated, accountingfor the reduction in size of both of the collapsed sub-groups.

In FIG. 9, all of the tracks of the composite presentation now fit inthe composite display area 615 concurrently, due to the collapsing ofthe two selected sub-groups. The composite display area still only showsa portion of the span of the timeline, as the horizontal axis is notaffected by the collapsing feature.

FIGS. 10-12 illustrate the expansion feature of some embodiments. FIG.10 illustrates the GUI 600 as shown in FIG. 9, after the application ofthe collapsing feature to sub-groups 635 and 720. FIG. 10 alsoillustrates cursor 630 over video effects sub-group 910.

FIG. 11 illustrates the GUI 600 after user interaction to select thevideo effects sub-group 910. The labels for the three rows in the videoeffects sub-group, rows 1105, 1110, and 1115 (V6, V7, and V8) are nowhighlighted.

FIG. 11 also illustrates the cursor 630 over the expansion tool 625.FIG. 12 illustrates the result of the user selecting the application ofthe expansion tool 625. The clip shapes in video effects sub-group 910have been enlarged in size vertically by a factor of approximately twoalong with the row heights for the three rows 1105, 1110, and 1115.Although the clip shapes expand in size vertically, the row assignmentsof the clip shapes are not lost because the rows are enlarged in sizecorrespondingly. Thus, the clip shapes are still displayed in a row thatcorresponds to the track to which the clip shapes' corresponding mediaclips are assigned. Furthermore, because the size of the clip shapeshorizontally is not affected, the association of each clip shape with aparticular portion of the timeline is not lost. A user that wishes toonly edit clip shapes in one particular group may expand that particulargroup in order to more precisely perform the desired editing operations.In some embodiments, as shown, expansion does not affect the row orgroup labeling.

Like the application of the collapsing tool, the editing applicationmust calculate new locations of all the clip shapes when the expansiontool is applied. The clip shapes in the main edit sub-group 905 andsupers sub-group 635 are not affected, because they are displayed abovethe video effects sub-group and have nowhere to move. New vertical spansfor each of the clip shapes in the video effects sub-group must becalculated based on the enlargement factor. The positions of the clipsshapes for the remainder of the sub-groups can then be calculated,accounting for the expansion of the video effects clip shapes.

In displaying the clip shapes in their new positions, some embodimentsanimate the movement of the clip shapes in the composite display area.For instance, some embodiments calculate the new positions, thendetermine how fast each of the shapes has to move or change size inorder to go from its initial position to final position. These speedsare then calculated in such a way that the animation appears smooth(i.e., as the clip shapes of the selected group are changing in size,the shapes underneath them are moving along in unison). In otherembodiments, however, there is no animation between the initial positionand the final position. Instead, the shapes are displayed at the finalposition immediately after the selection of the collapsing or expansiontool.

The collapsing and expansion tools provide users with the ability tode-emphasize or emphasize groups of clip shapes. Furthermore, thecollapsing tool enables more of the desired groups of clip shapes to beviewable in the composite display area concurrently. If a user wanted toedit, for example, the music sub-group 925 while viewing the clip shapesin the main edit sub-group 905 (e.g., to line up music with the desiredvideo), the user could collapse the clip shapes in all of the fivegroups in between those two groups.

III. Compressing Clip Shapes in the Composite Display Area

As noted above, the collapsing tool is one type of composite displayarea condensing tool provided by the media-editing application of someembodiments. Alternatively or conjunctively, some embodiments provide acompression tool that reduces blank space in the composite display areaby moving clip shapes together. This also results in the condensing ofthe clip shapes in the composite display area, such that more of theclip shapes are displayed at once in the composite display area.

FIG. 13 conceptually illustrates a process 1300 of some embodiments forapplying a compression feature. Process 1300 will be described byreference to FIGS. 14 and 15, which illustrate one possibleimplementation of the compression feature of some embodiments. FIGS. 14and 15 illustrate a GUI 1400 for a media-editing application. Similar tothe GUI 600 of FIG. 6, GUI 1400 includes a media library 1405, a previewdisplay area 1410, and a composite display area 1415. GUI 1400 alsoincludes a compression tool 1420.

As shown in FIG. 13, process 1300 begins (at 1305) by displaying acomposite display area that includes several clip shapes in initiallocations based on initial row assignments. Composite display area 1415is an example of such a composite display area. Each of the clip shapesin composite display area 1415 is in a row. In some embodiments, theinitial row represents a track to which the media clip represented bythe clip shape is actually assigned. The assignments to tracks of themedia clips is used by the media-editing application when creating thecomposite media presentation.

These tracks (and thus the rows) are arranged into groups in someembodiments, as shown in composite display area 1415. The rows (and thusthe clip shapes) are arranged into two primary groups, video and audio.The video group is split into three sub-groups and the audio group issplit into four sub-groups, like the groups in the composite displayarea 615 of FIGS. 6-12.

Next, process 1300 receives (at 1310) input to apply the compressionfeature. In some embodiments, the compression feature is provided as aUI item such as item 1420. The item may be a selectable button, commandin a pull-down, drop-down, or other type of menu, or another type of UIitem. In some embodiments, the compression feature may also be invokedby keyboard input. FIG. 14 illustrates a cursor 1425 over thecompression UI item 1420.

After receiving input to apply the compression feature, the processdetermines (at 1315) new row assignments for each clip shape. The newrow assignment for a particular clip shape indicates in which row in thecomposite display area the particular clip shape will be displayed. Thenew row assignment is merely a graphical assignment in some embodiments,and does not indicate that the corresponding media clip has actuallychanged tracks for the purpose of the actual creation of the mediapresentation. As such, after the application of the compression feature,the row assignments of the clip shapes are disassociated from the trackassignments of the corresponding media clips.

In general, in assigning new rows, the process attempts to push the clipshapes together vertically without actually overlapping any of the clipshapes. In some embodiments, a destination row is selected, and thetracks are all pushed towards the destination row. However, each clipshape remains spanning the same portion of the timeline as in itsinitial position. Furthermore, a clip shape cannot pass by another clipshape if two clip shapes share a portion of the timeline. The rowassignment process of some embodiments will be described in detail belowby reference to FIG. 20.

The process then determines (at 1320) new locations for each clip shape.In some embodiments, the new row assignment and the new location are thesame for each clip shape. However, in some embodiments, intermediaterows are removed (e.g., when groups and/or sub-groups are compressedseparately) such that entire groups of tracks are shifted upwards ordownwards as well. Thus, for some clip shapes, not only does a new rowassignment need to be determined, but the new y-coordinate location ofthat row must be accounted for as well.

In some embodiments, determining the new locations entails determiningnew locations in the composite presentation window (such as compositepresentation window 400 of FIG. 4). Once the new locations in thecomposite presentation window are determined for the clip shapes, thenthe new locations in the displayed composite display area are only amatter of determining what portion of the composite presentation windowis within the display window (e.g., window 410).

Once the new locations are determined, process 1300 then computes (at1325) a velocity for moving each graphical representation from itsinitial location to a new location based on the new row assignments. Insome embodiments, the movement of the clip shapes from the initiallocation to the new location is animated in the composite display area.The animation is such that each clip shape starts and stops moving atthe same time. Thus, the application determines the total distance eachclip shape is to move, then divides those distances over the timeallotted for the animation in order to compute the clip shapevelocities.

The process then displays (at 1330) the movement of the clip shapes fromtheir initial locations to their new locations. As noted, this movementis animated such that all of the clip shapes start and stop moving inunison in some embodiments. Some embodiments, though, do not animate theclip shapes and instead just immediately display the clip shapes in thenew locations. After displaying the clip shapes in the new locations,the process ends.

FIG. 15 illustrates the composite display area 1415 after theapplication of the compression feature to the clip shapes from FIG. 14.This illustrates the case where each of the seven groups is compressedseparately, and the clip shapes in each group remain within the separategroups. In the illustrated case, the destination row for each group isthe topmost row in the group. Some embodiments use the topmost row as adefault. Some embodiments allow a user to select a row (e.g., topmost,bottommost, middle row, etc.). Other possibilities for how compressionis applied (e.g., compressing all of the clips across the groups,compressing different groups in different directions) are describedbelow.

The clip shapes in the main edit group 1505 are not moved at all. Thisis the case because all of the clip shapes in the V2 row (using thetrack name to identify the row, as shown in FIG. 14) are prevented frommoving upwards into the V1 row by clip shapes in the V1 row. Forinstance, clip shape 1505 cannot move up a track because it partiallyoverlaps in the timeline with clip shape 1510.

On the other hand, the composite display area real estate occupied bythe main audio group 1520 is reduced from four rows to two rows by theapplication of the compression feature. Clip shapes 1525 and 1530 eachmove up two rows such that they are even vertically with clip shape1535. This enables clip shape 1540 to also move up two rows, as clipshapes 1525 and 1530 are no longer preventing this move. Similarly, clipshapes 1545 and 1550 each move up one row, and the movement of clipshape 1545 allows clip shape 1555 to move up two TOWS.

Each of the other groups (supers, video effects, dialog, audio effects,and music) is compressed from three rows of clip shapes to two rows.This enables all of the clip shapes to be displayed in full size in thecomposite display area, although this will not always be the case.Although the track data is lost visually in some embodiments (i.e., therows no longer are associated with a particular track), the associationof each clip shape with a particular portion of the timeline is notlost. Accordingly, a user may still perform editing operations on thecompressed clip shapes even though they are not displayed rows thatcorrespond to the tracks to which the corresponding media clips areassigned for the creation of the presentation.

When the groups are compressed, various aspects of the display of thecomposite display area are modified in some embodiments. As shown inFIG. 15, the row labels are removed in some embodiments because the rowsin the composite display area no longer correspond to particular tracks.For instance, clip shape 1530 and clip shape 1555 are not assigned tothe same track when the editing application creates the composite videopresentation, but they are displayed in the same row in the compositedisplay area 1415. Some embodiments also remove the row indicator linescompletely. On the other hand, some embodiments maintain the trackassociations visually by displaying track lines as moving up and down inthe composite display area with the clip shapes. For example, in suchembodiments, the track line between initial rows A2 and A3 would runbelow clips 1525 and 1530 (along with the track line between initialrows A1 and A2), then diagonal downwards and run between clips 1535 and1555 (along with the track line between initial rows A3 and A4), thencontinue on in between clips 1545 and 1550 (without any other tracklines). While displaying track lines, some embodiments neverthelessremove the track names (row labels) so that the beginning of more trackswill be visible in the composite display area.

FIGS. 14 and 15 illustrated the case in which each of the groups iscompressed separately. FIGS. 16 and 17 illustrate the case in which allof the clip shapes are compressed together and the group boundaries areremoved. FIG. 16 illustrates a GUI 1600 for a media-editing application.Like GUI 1400 of FIG. 14, GUI 1600 includes a media library 1605, apreview display area 1610, a composite display area 1615, and acompression UI item 1620. Again, the composite display area 1615 isarranged into the same groups and sub-groups as above. FIG. 16 alsoillustrates a cursor 1625 over the compression UI item 1620.

FIG. 17 illustrates the result of user interaction to apply thecompression feature (e.g., by pressing and holding a cursor controllerbutton with the cursor over the compression UI item 1620). In FIG. 17,all of the clip shapes are moved upwards towards row V1. The clip shapesin the first two rows have not moved, because all of the clip shapes inrow V2 are blocked by clip shapes in V1. However, the remainder of theclip shapes have at least moved up slightly. Clip shapes 1705 and 1710,for example, move up an amount equal to the thickness of the sub-groupboundary. Other clip shapes move up significantly further. For instance,clip shape 1715, from the main audio sub-group, moves up to the same rowas clip shapes 1705 and 1710.

In FIG. 17, the composite display area 1615 does not display any grouplabels or row labels. As the sub-groups are no longer separate, thegroup labels no longer have any meaning. Some embodiments, though,display the clip shapes of each sub-group in a different color. In theexample shown, the clip shapes of each of the primary groups (video andaudio) have different colors, but there is no distinction between thesub-groups within a group.

The track lines have also been removed from composite display area 1615in FIG. 17. As noted above, some embodiments remove track linesaltogether, some embodiments display the track lines as before but donot ascribe meaning to the track lines in terms of representing theactual data tracks, and some embodiments display the track lines fromthe uncompressed composite display area as moving up and down in thecomposite display area with the clip shapes.

An example of such track lines is illustrated in FIGS. 18 and 19. Thesefigures illustrate a composite display area 1800 and the application ofthe compression feature to the clip shapes in the composite displayarea. When the compression feature is applied to the composite displayarea 1800, the track lines between the rows of clip shapes are displayedas moving along with the compressed clip shapes.

FIG. 18 illustrates composite display area 1800 prior to compressionbeing applied to the display area. The display area includes multipleclip shapes among 10 rows, each of which correspond to a track. Betweenthe rows are nine track lines 1805-1845. FIG. 19 illustrates thecomposite display area 1800 after the application of the compressionfeature. The clip shapes have been compressed from ten rows into seven,where the vertical position of the rows no longer corresponds to aparticular track.

FIG. 19 also displays the track lines 1805-1845 between the clip shapes.Each of the track lines runs over or under the same clip shapes as inthe uncompressed composite display area of FIG. 18. For example, trackline 1805 starts in the same place, but after passing between clipshapes 1905 and 1910 it moves up to run above clip shape 1915 and thenback down below clip shape 1920, then continues on in a straight path.As a second example, track line 1835 is quickly merged with track lines1830 and 1840 to pass between clip shapes 1925 and 1930, continuesbetween clip shapes 1935 and 1930 while track line 1830 breaks offupwards, drops down underneath clip shape 1940, and finally splits offof track line 1840 at the right edge of the composite display area. Asshown, some embodiments split up any merged track lines at the edges ofthe composite display area in order to indicate the different tracks tothe user.

The track lines in FIG. 19 are illustrated as merged when multiple tracklines run between clip shapes. However, some embodiments do not mergethe track lines but instead spread the clip shapes out enough such thateach track line is visible separately. These slight adjustments in theclip shape position must be accounted for when determining the newpositions for the clip shapes during the compression process (e.g.,operation 1320 of process 1300).

IV. Row Assignment of Clip Shapes During Compression

As described above, when applying the compression feature, themedia-editing application of some embodiments determines new rows foreach of the clip shapes. The following section will describe thisprocess in greater detail. FIG. 20 conceptually illustrates a process2000 of some embodiments for assigning clip shapes to new rows whenapplying the compression feature to clip shapes in a composite displayarea. Process 2000 is applied to each group that is being compressed.Thus, if there are two groups, process 2000 is applied twice. In theexample of FIGS. 14 and 15, process 2000 is applied seven times, oncefor each sub-group. On the other hand, in the example of FIGS. 16 and17, process 2000 is applied only once to all of the clip shapes. One ofordinary skill will recognize that process 2000 is only one example of aprocess for assigning clip shapes to new rows when applying thecompression feature of some embodiments, and that other processes arepossible to serve this purpose as well.

Process 2000 will be described by reference to FIGS. 21 and 22. FIG. 21illustrates a GUI 2100 for a media-editing application that includes amedia library 2105, a preview display area 2110, a composite displayarea 2115, and a compression UI item 2120. These elements of the GUI2100 are similar to the corresponding elements of the GUIs describedabove, such as GUI 600. The composite display area 2115 includes twogroups, a video group 2125 and an audio group 2130. The application ofthe compression feature to composite display area 2115 causes the clipshapes in the video group to move downward and the clip shapes in theaudio group to move upward. FIGS. 22A-22D illustrate the application ofprocess 2000 to the composite display area 2115 in order to determinenew rows for the display of the clip shapes to apply the compressionfeature. FIGS. 22A-22D illustrates this process over the course of 10stages, 2201-2210. In FIGS. 22A-22D, the process is actually appliedtwice, first to the video group 2125 and then to the audio group 2130.

Process 2000 begins by determining (at 2005) an initial destination rowand setting this initial destination row as the current destination row.The initial destination row is the row towards which all of the clipshapes move in some embodiments. FIG. 22A conceptually illustrates adata construct 2215 for composite display area 2115. Row 2220 is theinitial destination track, and thus the first current destination track.FIG. 22A also conceptually illustrates a storage bin 2250. Storage bin2250 is a data storage (e.g., a data structure) that includes a markerfor each of the clip shapes in the data construct 2215. Once a clipshape is assigned to a row, the corresponding marker is removed from thestorage bin 2250. In some embodiments, removing a marker entailsindicating that the marker's clip shape has been assigned.

Process 2000 next assigns (at 2010) any unassigned clip shapes on thecurrent destination row to the current destination row. These are theclip shapes that are not changing rows during the compression process.At stage 2201 of FIG. 22A, the clip shapes V1 and V2 are assigned to therow 2220. These shapes are also removed from storage bin 2250 at thistime, as they are now assigned.

The process next determines (at 2015) whether there are any unassignedclip shapes. When all of the clip shapes are assigned, process 2000ends. Otherwise, the process defines (at 2020) a data structure for thecurrent destination row. The data structure indicates x-coordinateswithin the row (i.e., along the timeline) at which clip shapes can andcannot be moved into the row. If a clip shape that is a candidate formoving into the row occupies any x-coordinate that is already indicatedin the data structure, then the clip shape cannot be moved into thedestination row. Data structure 2255 of FIG. 22A is a conceptualillustration of such a data structure. Data structure 2255 is marked offwith Xs at x-coordinates that correspond to clip shapes V1 and V2. Whiledata structure 2255 illustrates fourteen x-coordinate regions, one ofordinary skill in the art will recognize that actual data structuresused will be much more finely split up in some embodiments.

Process 2000 then determines (at 2025) a current analysis row. This isthe row on which clip shapes will be analyzed for movement into thecurrent destination row. The first analysis row for a particulardestination row is the row that is next to the particular destinationrow, moving away from the initial destination row. Thus, in stage 2202of FIG. 22A, the current analysis row is row 2225.

The process then analyzes (at 2030) the clip shapes on the currentanalysis row to determine whether they can be assigned to the currentdestination row. In some embodiments, this entails determining, for aparticular clip shape on the current analysis row, whether the clipshape occupies any x-coordinate that is not available in the datastructure for the destination row.

The process then assigns (at 2035) any non-overlapping clip shapes onthe current analysis row to the current destination row. In stage 2202of FIG. 22A, clip shape V3 is such a non-overlapping clip shape, asthere is no indicator in the data structure 2255 at any of thex-coordinates occupied by clip shape V3. On the other hand, clip shapeV4 cannot move down to row 2220 because it is partially blocked by clipV1. Thus, clip shape V3 is assigned to row 2220 and clip shape V4remains unassigned. As clip shape V3 is assigned, its marker is removedfrom storage bin 2250.

Process 2000 next modifies (at 2040) the data structure for the currentdestination row to include information for all of the clip shapes thatwere in the current analysis row. This includes both clip shapes thatwere assigned to the current destination row and those that were not.The clip shapes that were overlapping, and thus not assigned to thedestination row, are nevertheless included because they can block clipshapes in analysis rows further from the destination row from movinginto the destination row. For instance, V4 could block any clip shapesabove from moving down to row 2220 and thus its presence must be notedin data structure 2255.

The process then determines (at 2045) whether any rows with unanalyzedclip shapes remain. That is, the process determines whether any rowsremain that could include clip shapes that move into the currentdestination row. When there are no more such rows, the process proceedsto 2055, which is described below. When at least one such row remains,the process increments (at 2050) the current analysis row by one row.Thus, at stage 2203 of FIG. 22A, the current analysis row has been movedup one to row 2230. Process 2000 then returns to 2030 to analyze theclip shapes in the current analysis row for movement to the currentdestination row. At stage 2203, clip shape V5 is not assigned todestination row 2220 because it is blocked by clip shapes V3 and V1,clip shape V6 is assigned to destination row 2220 because it fitsbetween clip shapes V4 and V2, and clip shape V7 is not assigned todestination row 2220 because it is blocked by clip shape V2. Becauseclip shape V6 is assigned, its corresponding marker is removed fromstorage bin 2250.

Once all possible rows have been analyzed for assignment to the currentdestination track, the process increments (at 2055) the currentdestination row by one row. This is the row that was the first analysisrow for the previous destination row. In the example of FIG. 22B, stage2204 illustrates that the destination row has now moved to row 2225.After incrementing the destination row, the process determines (at 2060)whether there are any unassigned clip shapes left, including clip shapesin the current destination row. When there are no more unassigned clipshapes, the process returns to 2010. Otherwise, the process ends.

Examining the storage bin 2250, clip shapes V4, V5, and V7 are allunassigned after stage 2203. Stage 2204 illustrates that clip shape V4is in current destination row 2225, and thus is assigned to this row(i.e., clip shape V4 does not move during the compression process). Thecorresponding marker for clip shape V4 is removed from the storage bin,and the x-coordinates occupied by the clip shape are indicated in thenew data structure 2260 for destination row 2225. Proceeding to stage2205, the initial analysis row for the current destination row 2225 isrow 2230. As shown, clip shapes V5 and V7 are both assigned todestination row 2225. Although clip shape V3 would be blocking clipshape V5, because clip shape V3 has been assigned to row 2220, it is nottaken into account when determining whether clip shape V5 can beassigned to row 2225. At this point, all of the video clip shapes instorage bin 2250 have been assigned to a new row. As such, the process2000 as applied to these clip shapes would end.

As noted, when compression is applied separately to more than one group,process 2000 or a similar process will be applied to each group. In theexample of FIGS. 22C-22D, stages 2206-2210 illustrate the application ofprocess 2000 to the audio group of clip shapes. In this example, theaudio clip shapes are moved up towards initial destination row 2235. Atstage 2206, the current destination row is row 2235, and the two clipshapes A1 and A2 in the row are assigned and removed from storage bin2250. Data structure 2265 is defined and indicators are entered to markthe x-coordinates of clip shapes A1 and A2.

At stage 2207, the first analysis row 2240 is analyzed for assignment todestination row 2235. Clip shape A3 cannot move up because of clip shapeA1, but clip shape A4 can move up and is thus assigned to row 2235 andremoved from storage bin 2250. The x-coordinates for both of these clipshapes are indicated in data structure 2265. At stage 2208, the analysisrow is moved to row 2245. Clip shape A5 is prevented from moving todestination row 2235 by clip shape A1, while clip shape A6 fits betweenclip shapes A2 and A4 and is thus assigned to row 2235 and removed fromstorage bin 2250.

At stage 2209, the destination row has moved to row 2240, and clip shapeA3 is assigned to this row and its marker removed from storage bin 2250.Data structure 2270 is defined for destination row 2240, and thex-coordinates of clip shape A3 are indicated in the data structure.Next, at stage 2210, row 2245 is the analysis row, and the onlyunassigned clip shape, clip shape A5, is assigned to destination row2240. When the marker for clip shape A5 is removed from storage bin2250, the storage bin is now empty and the row assignment process canend.

Once the rows are assigned, the editing application of some embodimentscan determine the velocities for animating the clip shapes and displaythe animation. FIG. 23 illustrates the animation of the compositedisplay area 2115 in three stages. The first stage 2310 illustrates theclip shapes at their initial positions, the second stage 2320illustrates the clip shapes approximately halfway through the animation,and the third stage 2330 illustrates the clip shapes at their finalpositions.

The first stage 2310 also conceptually illustrates the distances thateach clip shape will move during the animation. Clip shapes V3, V5, V7,A4, and A5 each move a distance 2305 (i.e., one row of movement). Clipshapes V6 and A6 each move a distance 2315 (i.e., two rows of movement,which is twice the distance 2305). One of ordinary skill will recognizethat in cases such as that illustrated in FIGS. 16 and 17, the distancesfor the animation will be more complicated due to the removal of groupand sub-group boundaries that take up space. Furthermore, if track linesare displayed, as illustrated in FIG. 19, space for the track lines mustbe factored into the distance calculations.

Stage 2320 illustrates the clip shapes in composite display area 2115approximately halfway through the animation of the compression process.At this point, clip shapes V6 and A6 have moved twice as far as clipshapes V3, V5, V7, A4, and A5. Arrows are shown to indicate thedirection that the clip shapes are moving at this point. The arrow headson the arrows for clip shapes V6 and A6 are larger than those for theother moving clip shapes to indicate that clip shapes V6 and A6 areanimated at a larger velocity.

Finally, stage 2330 illustrates the composite display area 2115 afterthe clip shapes have finished moving. In the case illustrated here, thetwo groups of clip shapes are moved towards the center of the compositedisplay area. While this compresses the clip shapes, it also leavesblank space above the upper of the two groups. Some embodiments shiftall of the clip shapes up one or more rows so as to avoid leaving blankspace in the composite display area. This can be important when thegroups are much larger and have significantly more tracks than are shownin the present example.

V. Software Architecture

In some embodiments, the processes described above are implemented assoftware running on a particular machine, such as a computer or ahandheld device, or stored in a computer readable medium. FIG. 24conceptually illustrates the software architecture of a media-editingapplication 2400 of some embodiments. In some embodiments, themedia-editing application is a stand-alone application or is integratedinto another application, while in other embodiments the applicationmight be implemented within an operating system. Furthermore, in someembodiments, the application is provided as part of a server-basedsolution. In some such embodiments, the application is provided via athin client. That is, the application runs on a server while a userinteracts with the application via a separate machine remote from theserver. In other such embodiments, the application is provided via athick client. That is, the application is distributed from the server tothe client machine and runs on the client machine.

Media-editing application 2400 includes a user interface (UI)interaction module 2405, an expansion module 2410, a collapsing module2415, a compression module 2420, an editing engine 2425, a compositedisplay area module 2430, a rendering engine 2435, and a previewgenerator 2437. The media-editing application also includes project datastorage 2455, content storage 2460, and other storages 2465. In someembodiments, the project data storage 2455 stores data about a compositemedia presentation, such as in and out points for media clips,information about transitions between media clips, etc. Content storage2460 includes the media clips that are used by the media-editingapplication to create a composite presentation. In some embodiments,storages 2455, 2460, and 2465 are all one physical storage. In otherembodiments, the storages are in separate physical storages, or two ofthe storages are in one physical storage while other storages are in adifferent physical storage.

FIG. 24 also illustrates an operating system 2470 that includes cursorcontroller driver 2475, keyboard driver 2480, and display module 2485.In some embodiments, as illustrated, the cursor controller driver 2475,keyboard driver 2480, and display module 2485 are part of the operatingsystem 2470 even when the media-editing application 2400 is anapplication separate from the operating system.

A user interacts with the user interface via input devices (not shown).The input devices, such as cursor controllers (mouse, tablet, touchpad,etc.) and keyboards, send signals to the cursor controller driver 2455and keyboard driver 2460, which translate those signals into user inputdata that is provided to the UI interaction module 2405. Someembodiments include a touchscreen that sends signals to the UIinteraction module 2405 as well. The UI interaction module interpretsthe user input data and passes it to various modules, including theexpansion module 2410, the collapsing module 2415, the compressionmodule 2420, and the editing engine 2425.

Expansion module 2410 receives input through the UI interaction module2405. When the input indicates to expand one or more groups, theexpansion module uses any necessary information from the project data2455 to determine which clip shapes to expand and how much to expandthem. The expansion module 2410 passes expansion information to thecomposite display area display module 2430, which generates the displayof the composite display area.

Similarly, collapsing module 2415 receives input through the UIinteraction module 2405. When the input indicates to collapse one ormore groups, the collapsing module uses any necessary information fromthe project data 2455 to determine which clip shapes to collapse and howmuch to collapse them. The collapsing module 2415 passes collapsinginformation to the composite display area display module 2430.

Compression module 2420 also receives input from the UI interactionmodule 2405. Compression module 2420 includes a row assignment module2440, a position calculator 2445, and a velocity calculator 2450. Insome embodiments, upon receiving input to compress clip shapes in acomposite display area, compression module 2420 performs process 1300 ora similar process. Compression module 2420 also receives any necessaryinformation (i.e., track assignment information) about the clip shapesfrom project data storage 2455.

The row assignment module 2440 assigns the clip shapes in the compositedisplay area to new rows. In some embodiments, module 2440 performsprocess 2000 or a similar process. The position calculator 2445 receivesthe row assignment information and calculates a new position in thecomposite display area for the clip shapes. The velocity calculator 2450calculates the speed and direction that each clip shape being compressedhas to move in the composite display area. This animation information isthen passed to the composite display area display module 2430.

The editing engine 2425 also receives information from the UIinteraction module 2405. A user can user the interface of the editingapplication to edit the composite media presentation through thecomposite display area. For instance, a user can modify the compositepresentation by using roll edits, ripple edits, slide edits, etc. Theediting engine 2425 passes information to the composite display area2430, and when the application is to render the media presentation, therendering engine 2435.

Composite display area display module 2430 manages the display of thecomposite display area of the GUI of the editing application. In someembodiments, module 2430 receives input from the UI interaction module2405 and manages group selection. In other embodiments, this isperformed by a separate module. Module 2430 also receives projectinformation from storage 2455 in order to determine what to display inthe composite display area. The composite display area may be modifieddue to information from the expansion module 2410, collapsing module2415, and/or compression module 2420. Edits performed by the editingengine will also affect the composite display area. Information aboutdisplaying the composite display area is sent to the display module2485.

Rendering engine 2435 enables the storage or output of audio and videofrom the media-editing application 2400. Rendering engine 2435 receivesdata from the editing engine 2425 and, in some embodiments creates acomposite media presentation from individual media clips. The compositemedia presentation can be stored in the storages or output to thedisplay module 2485.

Preview generator 2437 enables the output of audio and video from themedia-editing application so that a user can preview the compositepresentation. The preview generator 2437, based on information from theediting module 2425 (and, in some embodiments, other modules), sendsinformation about how to display each pixel of a presentation to thedisplay module 2485.

While many of the features have been described as being performed by onemodule (e.g., the expansion module 2410 or collapsing module 2415), oneof ordinary skill would recognize that the functions might be split upinto multiple modules, and the performance of one feature might evenrequire multiple modules.

FIG. 25 conceptually illustrates a process 2500 of some embodiments formanufacturing a computer readable medium that stores a media-editingapplication such as the application 2400 described above. In someembodiments, the computer readable medium is a distributable CD-ROM. Asshown, process 2500 begins by defining (at 2505) a composite displayarea for displaying graphical representations of a set of media clips.For instance, the composite display areas 320 and 615 of FIGS. 3 and 6are examples of the defined composite display area. The process nextdefines (at 2510) a set of groups for arranging the graphicalrepresentations of media clips in the composite display area. The groupsand sub-groups 355-364 in composite display area 320 are one example ofthe groups that could be defined. Process 2500 then defines (at 2515) adisplay area for displaying a composite presentation created bycompositing the set of media clips. For instance, preview display area310 is such a display area.

Process 2500 then defines (at 2520) a compression user interface toolfor invoking a compression feature. The compression feature described byreference to FIGS. 13-23 above is an example of such a feature. Theprocess next defines (at 2525) a compression module for assigninggraphical representations to new locations in the composite display areain order to reduce blank space (i.e., for executing the compressionfeature). The module 2420 of FIG. 24 above is an example of such acompression module.

Process 2500 then defines (at 2530) a collapsing UI item for invoking acollapsing feature and defines (at 2535) an expansion UI item forinvoking an expansion feature.

The collapsing and expansion features described by reference to FIGS.5-12 above are examples of such features. The process next defines (at2540) default, collapsed, and expanded graphical representations forclips. In some embodiments, these are the standard size in a compositedisplay area for the graphical representations and the sizes of thegraphical representations after collapsing and expanding them. Next,process 2500 defines (at 2545) a collapsing module for reducing the sizeof graphical representations for at least one of the groups in thecomposite display area. The process defines (at 2550) an expansionmodule for increasing the size of the graphical representations for atleast one of the groups in the composite display area. Collapsing module2415 and expansion module 2410 are examples of such modules.

The process then defines (at 2552) other media editing tools andfunctionalities. Examples of such editing tools may include zoom, colorenhancement, blemish removal, audio mixing, trim tools, etc. Inaddition, various other media editing functionalities may be defined.Such functionalities may include library functions, format conversionfunctions, etc. The process defines these additional tools in order tocreate a media editing application that has many additional features tothe features described above.

A more detailed view of a media editing application with theseadditional features is illustrated in FIG. 26. Specifically, this figureshows a media editing application with these additional tools. FIG. 26illustrates a list of video and/or audio clips 2610, video editing tools2620, and video displays 2630. The list of clips 2610 includes videoclips along with metadata (e.g., timecode information) about the videoclips. In some embodiments, the list of video clips is the list of videoclips in a particular sequence of video clips, and the metadataspecifies in and out points, durations, etc. for the video clips.

The video editing tools 2620 include tools that allow a user tographically set in and out points for video clips (in other words, wherein the final product a specific clip or part of a clip will be shown).The video editing tools 2620 can be used to modify the temporal sequenceof the video frame and to synchronize audio tracks with video tracks(e.g., in order to add music over a video clip). In some embodiments,video editing tools 2620 also give users the ability to edit in effectsor perform other video editing functions. In some embodiments, the videoediting tools include trim tools for performing edits such as slideedits, ripple edits, slip edits, roll edits, etc.

Video displays 2630 allow a user to watch multiple video clips at once,thereby enabling easier selection of in and out points for the videoclips. The screen shot 2600 illustrates a few of many different editingtools that a video editing application of some embodiments may have toedit digital video.

In some cases, some or all of the video clips that are displayed in thelist of clips 2610, played in displays 2630, and edited by a user withvideo editing tools 2620, are video clips of real-world objects (e.g.,people, landscapes, etc.) filmed by a camera and include real-worldaudio (e.g., conversations, real-world noises, etc.) recorded by acamera, microphone, etc. In some cases, some or all of the video clipsare computer-generated animations or include computer generatedanimations (e.g., animated objects, computer-generated effects, etc.).

Returning to FIG. 25, process 2500 next stores (at 2555) the definedelements (i.e., the defined modules, UI items, etc.) on a computerreadable storage medium. As mentioned above, in some embodiments thecomputer readable storage medium is a distributable CD-ROM. In someembodiments, the medium is one or more of a solid-state device, a harddisk, a CD-ROM, or other non-volatile computer readable storage medium.

One of ordinary skill in the art will recognize that the variouselements defined by process 2500 are not exhaustive of the modules,rules, processes, and UI items that could be defined and stored on acomputer readable storage medium for a media editing applicationincorporating some embodiments of the invention. In addition, theprocess 2500 is a conceptual process, and the actual implementations mayvary. For example, different embodiments may define the various elementsin a different order, may define several elements in one operation, maydecompose the definition of a single element into multiple operations,etc. In addition, the process 2500 may be implemented as severalsub-processes or combined with other operations within a macro-process.

VI. Computer System

Many of the above-described features and applications are implemented assoftware processes that are specified as a set of instructions recordedon a computer readable storage medium (also referred to as computerreadable medium). When these instructions are executed by one or morecomputational element(s) (such as processors or other computationalelements like ASICs and FPGAs), they cause the computational element(s)to perform the actions indicated in the instructions. Computer is meantin its broadest sense, and can include any electronic device with aprocessor. Examples of computer readable media include, but are notlimited to, CD-ROMs, flash drives, RAM chips, hard drives, EPROMs, etc.The computer readable media does not include carrier waves andelectronic signals passing wirelessly or over wired connections.

In this specification, the term “software” is meant to include firmwareresiding in read-only memory or applications stored in magnetic storagewhich can be read into memory for processing by a processor. Also, insome embodiments, multiple software inventions can be implemented assub-parts of a larger program while remaining distinct softwareinventions. In some embodiments, multiple software inventions can alsobe implemented as separate programs. Finally, any combination ofseparate programs that together implement a software invention describedhere is within the scope of the invention. In some embodiments, thesoftware programs when installed to operate on one or more computersystems define one or more specific machine implementations that executeand perform the operations of the software programs.

FIG. 27 illustrates a computer system with which some embodiments of theinvention are implemented. Such a computer system includes various typesof computer readable media and interfaces for various other types ofcomputer readable media. Computer system 2700 includes a bus 2705, aprocessor 2710, a graphics processing unit (GPU) 2720, a system memory2725, a read-only memory 2730, a permanent storage device 2735, inputdevices 2740, and output devices 2745.

The bus 2705 collectively represents all system, peripheral, and chipsetbuses that communicatively connect the numerous internal devices of thecomputer system 2700. For instance, the bus 2705 communicativelyconnects the processor 2710 with the read-only memory 2730, the GPU2720, the system memory 2725, and the permanent storage device 2735.

From these various memory units, the processor 2710 retrievesinstructions to execute and data to process in order to execute theprocesses of the invention. In some embodiments, the processor comprisesa Field Programmable Gate Array (FPGA), an ASIC, or various otherelectronic components for executing instructions. In some embodiments,the processor Some instructions are passed to and executed by the GPU2720. The GPU 2720 can offload various computations or complement theimage processing provided by the processor 2710. In some embodiments,such functionality can be provided using CoreImage's kernel shadinglanguage.

The read-only-memory (ROM) 2730 stores static data and instructions thatare needed by the processor 2710 and other modules of the computersystem. The permanent storage device 2735, on the other hand, is aread-and-write memory device. This device is a non-volatile memory unitthat stores instructions and data even when the computer system 2700 isoff. Some embodiments of the invention use a mass-storage device (suchas a magnetic or optical disk and its corresponding disk drive) as thepermanent storage device 2735.

Other embodiments use a removable storage device (such as a floppy disk,flash drive, or ZIP® disk, and its corresponding disk drive) as thepermanent storage device. Like the permanent storage device 2735, thesystem memory 2725 is a read-and-write memory device. However, unlikestorage device 2735, the system memory is a volatile read-and-writememory, such a random access memory. The system memory stores some ofthe instructions and data that the processor needs at runtime. In someembodiments, the invention's processes are stored in the system memory2725, the permanent storage device 2735, and/or the read-only memory2730. For example, the various memory units include instructions forprocessing multimedia items in accordance with some embodiments. Fromthese various memory units, the processor 2710 retrieves instructions toexecute and data to process in order to execute the processes of someembodiments.

The bus 2705 also connects to the input and output devices 2740 and2745. The input devices enable the user to communicate information andselect commands to the computer system. The input devices 2740 includealphanumeric keyboards and pointing devices (also called “cursor controldevices”). The output devices 2745 display images generated by thecomputer system. For instance, these devices display a GUI. The outputdevices include printers and display devices, such as cathode ray tubes(CRT) or liquid crystal displays (LCD).

Finally, as shown in FIG. 27, bus 2705 also couples computer 2700 to anetwork 2765 through a network adapter (not shown). In this manner, thecomputer can be a part of a network of computers (such as a local areanetwork (“LAN”), a wide area network (“WAN”), or an Intranet, or anetwork of networks, such as the internet. For example, the computer2700 may be coupled to a web server (network 2765) so that a web browserexecuting on the computer 2700 can interact with the web server as auser interacts with a GUI that operates in the web browser.

Any or all components of computer system 2700 may be used in conjunctionwith the invention. For instance, in some embodiments the execution ofthe frames of the rendering is performed by the GPU 2720 instead of theCPU 2710. Similarly, other image editing functions can be offloaded tothe GPU 2720 where they are executed before the results are passed backinto memory or the processor 2710. However, a common limitation of theGPU 2720 is the number of instructions that the GPU 2720 is able tostore and process at any given time. Therefore, some embodiments adaptinstructions for implementing processes so that these processes fit ontothe instruction buffer of the GPU 2720 for execution locally on the GPU2720. Additionally, some GPUs 2720 do not contain sufficient processingresources to execute the processes of some embodiments and therefore theCPU 2710 executes the instructions. One of ordinary skill in the artwould appreciate that any other system configuration may also be used inconjunction with the present invention.

Some embodiments include electronic components, such as microprocessors,storage and memory that store computer program instructions in amachine-readable or computer-readable medium (alternatively referred toas computer-readable storage media, machine-readable media, ormachine-readable storage media). Some examples of such computer-readablemedia include RAM, ROM, read-only compact discs (CD-ROM), recordablecompact discs (CD-R), rewritable compact discs (CD-RW), read-onlydigital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), a varietyof recordable/rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.),flash memory (e.g., SD cards, mini-SD cards, micro-SD cards, etc.),magnetic and/or solid state hard drives, read-only and recordableblu-ray discs, ultra density optical discs, any other optical ormagnetic media, and floppy disks. The computer-readable media may storea computer program that is executable by at least one processor andincludes sets of instructions for performing various operations.Examples of hardware devices configured to store and execute sets ofinstructions include, but are not limited to application specificintegrated circuits (ASICs), field programmable gate arrays (FPGA),programmable logic devices (PLDs), ROM, and RAM devices. Examples ofcomputer programs or computer code include machine code, such as isproduced by a compiler, and files including higher-level code that areexecuted by a computer, an electronic component, or a microprocessorusing an interpreter.

As used in this specification and any claims of this application, theterms “computer”, “server”, “processor”, and “memory” all refer toelectronic or other technological devices. These terms exclude people orgroups of people. For the purposes of the specification, the termsdisplay or displaying means displaying on an electronic device. As usedin this specification and any claims of this application, the terms“computer readable medium” and “computer readable media” are entirelyrestricted to tangible, physical objects that store information in aform that is readable by a computer. These terms exclude any wirelesssignals, wired download signals, and any other ephemeral signals.

While the invention has been described with reference to numerousspecific details, one of ordinary skill in the art will recognize thatthe invention can be embodied in other specific forms without departingfrom the spirit of the invention. Thus, one of ordinary skill in the artwould understand that the invention is not to be limited by theforegoing illustrative details, but rather is to be defined by theappended claims.

1-23. (canceled)
 24. A non-transitory computer readable medium storing amedia editing application for creating multimedia presentations, theapplication comprising a graphical user interface (GUI), the GUIcomprising: a display area for displaying a preview of a compositepresentation that the application creates by compositing a plurality ofmultimedia clips; a timeline that represents a duration of the compositepresentation; a composite display area comprising a plurality of tracksthat span the timeline to display graphical representations of theplurality of multimedia clips that are part of the compositepresentation, wherein a plurality of tracks display a group of graphicalrepresentations that each spans a portion of the duration; a selectiontool for selecting a group to de-emphasize in the composite displayarea; and a collapsing tool for reducing, in a direction that isorthogonal to the timeline, a size of each graphical representation inthe selected group while maintaining the graphical representation's spanwith respect to the timeline.
 25. The non-transitory computer readablemedium of claim 24, wherein the direction is a vertical direction,wherein the collapsing tool is for reducing the vertical size of thegraphical representations.
 26. The non-transitory computer readablemedium of claim 24, wherein reducing the size of the graphicalrepresentations for a particular selected group comprises reducing thesize of the tracks that display the selected group of clip graphicalrepresentations.
 27. The non-transitory computer readable medium ofclaim 24, wherein the direction is a vertical direction and theplurality of tracks span the timeline in a horizontal direction, whereinthe collapsing tool reduces the size of the graphical representations byreducing a vertical size of the graphical representations whilemaintaining a horizontal size of the graphical representations.
 28. Thenon-transitory computer readable medium of claim 27, wherein thehorizontal size of each graphical representation corresponds to aparticular length of time within the composite presentation.
 29. Thenon-transitory computer readable medium of claim 24, wherein the groupis a video group and the plurality of tracks are video tracks, whereinthe composite display area comprises a plurality of audio tracks thatdisplay graphical representations for an audio group, wherein thecollapsing tool is further for selecting one or more groups in order toreduce the size of graphical representations in each of the selectedgroups.
 30. The non-transitory computer readable medium of claim 29,wherein the audio group comprises a main audio sub-group, a dialogsub-group, an audio effects sub-group, and a music sub-group.
 31. Thenon-transitory computer readable medium of claim 29, wherein the videogroup comprises a main edit sub-group, a supers sub-group, and a videoeffects sub-group.
 32. The non-transitory computer readable medium ofclaim 29, wherein the GUI further comprises an expansion tool forincreasing the size of the graphical representations for at least one ofthe groups in the composite display area.
 33. A method of storing amedia editing application for creating multimedia presentations, themethod comprising: providing a display area for displaying a preview ofa composite presentation that the application creates by compositing aplurality of multimedia clips; providing a timeline that represents aduration of the composite presentation; providing a composite displayarea comprising a plurality of tracks that span the timeline to displaygraphical representations of the plurality of multimedia clips that arepart of the composite presentation, wherein a plurality of tracksdisplay a group of graphical representations that each spans a portionof the duration; providing a selection tool for selecting a group tode-emphasize in the composite display area; and providing a collapsingtool for reducing, in a direction that is orthogonal to the timeline, asize of each graphical representation in the selected group whilemaintaining the graphical representation's span with respect to thetimeline.
 34. A method of claim 33, wherein the direction is a verticaldirection, wherein the collapsing tool is for reducing the vertical sizeof the graphical representations.
 35. The method of claim 33, whereinreducing the size of the graphical representations for a particularselected group comprises reducing the size of the tracks that displaythe selected group of clip graphical representations.
 36. The method ofclaim 33, wherein the direction is a vertical direction and theplurality of tracks span the timeline in a horizontal direction, whereinthe collapsing tool reduces the size of the graphical representations byreducing a vertical size of the graphical representations whilemaintaining a horizontal size of the graphical representations.
 37. Themethod of claim 36, wherein the horizontal size of each graphicalrepresentation corresponds to a particular length of time within thecomposite presentation.
 38. The method of claim 33, wherein the group isa video group and the plurality of tracks are video tracks, wherein thecomposite display area comprises a plurality of audio tracks thatdisplay graphical representations for an audio group, wherein thecollapsing tool is further for selecting one or more groups in order toreduce the size of graphical representations in each of the selectedgroups.
 39. The method of claim 38, wherein the audio group comprises amain audio sub-group, a dialog sub-group, an audio effects sub-group,and a music sub-group.
 40. The method of claim 38, wherein the videogroup comprises a main edit sub-group, a supers sub-group, and a videoeffects sub-group.
 41. The method of claim 38 further comprisingproviding an expansion tool for increasing the size of the graphicalrepresentations for at least one of the groups in the composite displayarea.